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Abstract 

An experiment was conducted to quantify the 
effects of changing dynamics on a subject’s ability to 
track a signal in order to eventually model a pilot 
adapting to changing aircraft dynamics. The data 
will be used to identify primary aircraft dynamics 
variables that influence changes in pilot’s response 
and produce a simplified pilot model that 
incorporates this relationship. Each run incorporated 
a different set of second-order aircraft dynamics 
representing short period transfer function pitch 
attitude response: damping ratio, frequency, gain, 
zero location, and time delay. The subject’s ability to 
conduct the tracking task was the greatest source of 
root mean square error tracking variability. As for 
the aircraft dynamics, the factors that affected the 
subjects’ ability to conduct the tracking were the time 
delay, frequency, and zero location. In addition to 
creating a simplified pilot model, the results of the 
experiment can be utilized in an advisory capacity. A 
situation awareness/prediction aid based on the pilot 
behavior and aircraft dynamics may help tailor pilot’s 
inputs more quickly so that PIO or an upset condition 
can be avoided. 

Introduction 


control law design. The research presented here 
builds on past work to provide tools to analyze and 
solve this type of modeling problem. 

Method 

An experiment was conducted to quantify the 
effects of changing dynamics on a subject’s ability to 
track a signal in order to eventually model a pilot 
adapting to changing aircraft dynamics. This paper 
documents the development of a database that will be 
used for this work. The experiment evaluated 
primary aircraft dynamics variables that influence 
changes in pilot’s response and produce a simplified 
pilot model that incorporates this relationship. 

The experiment, conducted to identify the pilot- 
dynamic variable relationship, required each subject 
to track as closely as possible a pitch attitude signal. 
The input for the tracking task was designed to 
contain a variety of frequencies in the frequency 
range associated with human pilot inputs, namely 0.1 
to 10 rad/sec. Each run incorporated a different set of 
second-order aircraft dynamics representing short 
period transfer function pitch attitude response. This 
response was then displayed to the pilot as the 
aircraft response. 


Significant research has been conducted to 
model or identify the pilot, as a way to quantify 
handling qualities or to better understand the 
behavior of a human pilot in controlling a vehicle [1- 
7]. Our research is investigating new analytical 
methods to model the pilot’s changing behavior over 
short time periods in response to changing aircraft 
dynamics. 

A model that captures the pilot’s adaptability to 
changing aircraft dynamics without a priori 
knowledge of these changes is the long-term 
objective of this research. Availability of such an 
analytical model would have impact in a number of 
different areas; among these are decision aids for the 
pilot, function allocation between the pilot and 
automation especially during changes in effective 
dynamics, and potential requirements for adaptive 


Independent Variables 

The following independent variables were 
controlled during each data run. 

Pitch Dynamics 

This experiment looked at short period 
dynamics, which affect flying qualities the most. 
Therefore, longitudinal dynamics were controlled 
while lateral/directional and thrust values were held 
constant at a heading of 40 deg and a speed of 300 
kts. The altitude at the beginning of each data run 
was 10,000 ft. The aircraft short-period pitch attitude 
transfer function was: 
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where 9 is pitch attitude and 8 is pilot stick. The 
damping ratio (g) was 0.4, 0.7, or 1. The short 
period frequency (at) was 0.5 Hz, 1 Hz, or 1.5 Hz. 
Gain ( K ) was 1, 2, or 3. Zero location ( L a ) was 

0.5, 1, or 1.5. Lastly, time delay T was 0 ms, 75 ms, 
or 150 ms. 


Tracking Signal 

The maneuver was designed as a 30 sec 
longitudinal tracking task. The tracking command 
input contained a range of frequencies and 
amplitudes to excite an appropriate dynamic range. 

The longitudinal tracking task was designed to 
contain a variety of discrete frequencies in the 
frequency range associated with human pilot inputs, 
namely 0.1 to 10 rad/sec. The input design method is 
described in detail by Klein and Morelli [8, 9]. The 
result was the equivalent of a frequency sweep input, 
but used many sinusoidal components with discrete 
frequencies applied simultaneously, instead of a 
single sinusoid with frequency increasing 
monotonically in time. 


Figure 1 shows an example tracking task. The 
sum-of-sines task was generated as: 
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where u(t) is the pitch attitude tracking command 
(deg), the discrete frequencies Ijrk/T were chosen 
to span the frequency range of 0.1 to 10 rad/sec, the 
phase angles { (/) k ) were random, and the amplitudes 

(A k ) were chosen to customize the power spectrum 

of the multi-sine input. In this application, the phase 
angles were chosen at random to emulate a random 
sinusoidal input. The frequency indices ( k ) were 
chosen at irregular intervals, which also helps to 
emulate a random sinusoidal input. 


Subjects 

Four pilots participated as subjects. The average 
age was 50 (standard deviation of 3.6) years old. The 
average years flying was 20.5±12 and the average 
number of flight hours was 3312±4483 hrs. Subject 
2 had the least number of flight hours (<600 hrs) and 
years flying (~ 6) whereas the next closest subject 
had just about 1000 flight hours and approximately 
15 years of flying experience. Subject 4 had the most 
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Figure 1. Longitudinal Input Signal 

number of flight hours (>10,000 hrs) and the most 
years flying (> 30 years) while the next nearest 
subject had just under 2000 flight hours. 

Dependent Variables 

During each of the 35 data runs, subjects tracked 
the pitch command input signal as closely as possible 
for 30 sec. After each run, subjects then completed a 
Cooper-Harper (CH) rating scale. 

RMS Error 

Root Mean Square (RMS) error was calculated 
by taking the current pitch attitude of the aircraft and 
subtracting the desired pitch attitude of the aircraft. 
This was calculated in deg. 

CH Rating 

At the end of each data run, subjects gave a 
modified CH rating [10, 11]. An electronic format of 
the CH rating process was used which enforced the 
structure of the CH logic decision path, starting at the 
bottom (“Is it controllable?”) format (Fig. 2), and 
progressing accordingly until a final rating is given 
[12]. The pitch attitude command signal ( u ) drove 
the flight director symbol on the Attitude Direction 
Indicator (ADI) shown in Figure 3. Zero tracking 
error was defined by the airplane symbol nudged 




Figure 2. Cooper-Harper Controllability Scale Format 



Figure 3. CH Optimal Rating Performance 


under the flight director (Fig. 3). Desired 

performance was defined as being within 2.5 degrees, 
which was half the width of the airplane symbol 
wings, of the flight director (Fig. 4). Adequate 
performance was defined as being within 5 degrees 
of the flight director which was the width of the 
airplane symbol wings (Fig. 5). 

Procedure 

After arrival, subjects were briefed on the 
experiment, task, and displays using still pictures. 



Figure 4. CH Desired Rating Performance 

Next, subjects were introduced to the simulator 
using a 20-minute familiarization run. The 
familiarization run had a 5 min periodic longitudinal 
tracking signal similar to the data runs. For both the 
familiarization run and for the data runs, the flight 
director had an amplitude of no more than ±5 
degrees. During the familiarization run, the subject 
was also able to free fly the simulation. The pitch 
attitude response transfer function (see Eq. 1) had the 
baseline characteristics of: £ = 0.4, co= 0.5 Hz, x = 0 
ms, L a = 1.5, and K = 3. Subjects were allowed to fly 












Figure 5. CH Adequate Rating Performance 


the familiarization run until they were comfortable 
with the simulation dynamics. 

After the subject was comfortable with the 
sidestick and the task, each subject completed 35 data 
runs. During each data run, straight and level flight 
was established for 2 sec before the tracking task 
initiated. After the 30 sec tracking task, straight and 
level flight was reestablished for 2 sec. Therefore, 
each run lasted a total of 34 sec. At the end of the 
runs, subjects gave a CH rating. 

The simulation was implemented in 
Matlab/Simulink® 2009b. The simulator setup is 
shown in Figure 6 and the general simulation 
diagram is shown in Figure 7. 

Results 

Analyses were done using SPSS™ vl6. 
Significance was set at a p-value < 0.05. 

Subject Effects on RMS Error 

Individual subjects tracking variability had the 
greatest effect on RMS error (F( 3>2 oi) = l 1.418, 
p<0.01). These data are shown in Figure 8 as the 
mean RMS error for each subject collapsed across all 



Figure 6. Simulator Setup 


aircraft dynamics variations with deviation bars, 
indicating 1 unit of computed standard error (SE) 
from the mean. A closer look at the data indicated 
that the subject with the larger RMS error had less 
flight hours (Subject 2 flight hours < 600) as 
compared to the other subjects who had more flight 
hours and a smaller RMS error. 


Aircraft Dynamic Effects on RMS Error 

Tables 1 and 2 detail how the aircraft dynamics 
affected the RMS error. As seen in Table 1, all 
individual aircraft dynamics significantly affected the 
RMS error. The most significant effects were the 
time delay, frequency, and zero location as indicated 
by rj 2 (Table 1). 

Time Delay Effects on RMS Error 

For the 150 ms time delay, subjects’ RMS error 
was significantly larger than for the 0 and 75 ms time 
delay (Fig. 9). This increase in RMS error was also 
reflected by the subjects’ CH rating. As can be seen 
in Figure 10, as the time delay increased, the vehicle 
moved towards more Level 2 and 3 handling 
qualities. 



Figure 7. Block Diagram of the Simulation 
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Figure 9. RMS Error by Time Delay 


Table 1. Effects of Dynamics on RMS Error 


Aircraft Dynamic 

F ( 1 , 207 ) 

P 

II 2 

Gain [K1 

9.64 

<0.01 

0.023 

Zero Location [L a ] 

26.12 

<0.01 

0.063 

Damping Ratio [Cl 

6.95 

<0.01 

0.017 

Frequency [co] 

58.20 

<0.01 

0.141 

Time Delay [x] 

14.99 

<0.01 

0.036 


Table 2. RMS Error by Aircraft Dynamics 


Aircraft Dynamic 

Mean RMS 
Error (deg) 

SE 

Gain [K] 

1 

1.86 

0.04 


2 

1.75 

0.05 


3 

2.40 

0.17 

Zero 

0.5 

1.74 

0.35 

Location 

1.0 

1.75 

0.38 

[La] 

1.5 

2.28 

1.21 

Damping 

0.4 

2.19 

0.12 

Ratio [Q 

0.7 

1.70 

0.05 

1.0 

1.90 

0.05 

Frequency 

0.5 

2.46 

1.22 

(Hz) [co] 

1.0 

1.70 

0.32 


1.5 

1.64 

0.24 

Time 

0 

1.85 

0.05 

Delay 

75 

1.70 

0.05 

(ms) [t] 

150 

2.28 

0.13 



Figure 10. CH Rating Count by Time Delay 


Frequency Effects on RMS Error 

The RMS error for the frequency increased as 
the speed of the system response decreased down to 
0.5 Hz (Fig. 11) which was different from the 1.0 and 
1.5 Hz frequencies. As with the time delay, subjects’ 
CH ratings also indicated this increase in RMS error 
(Fig. 12). The handling qualities shifted to Level 2 
and 3 as the frequency decreased. 




Figure 1 1 . RMS Error by Frequency Figure 1 3 . RMS Error by Zero Location 




CH Rating 


Figure 12. CH Rating Count by Frequency 

Zero Location Effects on RMS Error 

Lastly, RMS error increased with a zero location 
further into the left half plane (Fig. 13). Once again, 
this increase in RMS error manifests itself in the 
subjects’ CH ratings. As the zero location moves 
farther into the left-half plane, the subjects’ CH rating 
spread increases towards Level 2 and 3 handling 
qualities (Fig. 14). Furthermore, this increase in 
RMS error was even more pronounced with a zero 
location further into the left half plane for lower 
frequencies (Fig. 15) as would be expected by the 
control anticipation parameter (CAP) which is 


Figure 14. CH Rating Count by Zero Location 

(D 2 

proportional to — [13, 14]. In fact, the interaction 

L a 

between the zero location and frequency was 
significant (F(i j2 07 ) = 10.001, p<0.01) and rj 2 was 
0.0242. 

Aircraft-Pilot Coupling 

When the system response was slower than the 
subject desired by either an increasing time delay or 
decreasing the frequency, especially with a 

corresponding zero location increase, the RMS error 
increased. In fact, some subjects mentioned 



Figure 15. RMS Error by Frequency and Zero 

that certain runs were “PIO prone” (Table 3). These 
runs had a frequency of 0.5 Hz with the zero location 
farther into the left half plane or a large time delay. 


Table 3. PIO Comments 


Time 
Delay 
(ms) [t] 

Frequency 
(Hz) [o>] 

Zero 

Location 

[L„] 

Count of 
Subjects 
Mentioning 
“PIO Prone” 

0 

0.5 

1.0 

i 

0 

0.5 

1.5 

2 

150 

0.5 

0.5 

4 

150 

0.5 

1.5 

6 


Moreover, subjects’ CH ratings became more 
diverse towards higher CH ratings as more subjects 
mentioned “PIO Prone” (Fig. 16). 

Pilot Modeling Considerations 

Using System Identification to Determine Hess 
Simplified Pursuit Model Gain Parameters 

The Hess simplified pursuit pilot model includes 
two gains, K p and K r and a fixed neuromuscular 
model to represent a pilot’s response to a vehicle [15, 
16]. K r is the gain that results in a minimum damping 
ratio of 0.15 and K p is the gain that provides the 
desired open-loop crossover frequency consistent 
with the classical McRuer cross-over model [6, 7]. 
The pilot- vehicle model is shown in Figure 17 where 
the vehicle is represent by Equation 1 and G nm , the 
neuromuscular model, is 



CH Rating 


Figure 16. CH Rating Count for PIO Conditions 



Figure 17. Hess Simplified Pursuit Model 


nm s 2 + 2(0.707)1 0s + 10 2 

For the purposes of this experiment, input C is 
commanded pitch attitude u , the output M is aircraft 

pitch attitude 6 and M is pitch rate q. The above 
model does not include a time delay. For this 
analysis, the time delay was incorporated into the 
Hess simplified pursuit model with a sixth order Pade 
approximation. 

System identification techniques were used to 
calculate these gains [9] from the recorded data runs. 
The K p and K r gains vary slightly by pilot and are 
dependent on the scenarios which had varying system 
dynamics (f(K,L a ,g,co,T) ) (Figs. 18 and 19). The 

dependency on the subject may be due to learning 
affects or piloting styles. 


Table 4. Repetition Effects on Kp and Kr 



Figure 18. Effects of Scenario and Subject on Kp 
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Figure 19. Effects of Scenario and Subject on Kr 

Furthermore, within a given subject and constant 
system dynamics, the calculated K p and K r values 
using system identification techniques varied by run 
(Table 4). The values typically settled around an 
average but there were some outliers. Therefore, the 
need for real-time system identification is apparent 
especially if the aircraft dynamics change. 

Lastly, the Hess simplified pursuit model was 
shown to generally emulate the recorded pilot 
behavior when using the K p and K r values calculated 
using system identification (Fig. 20 where K= 2, L a = 1, 
C= 0.7, co= 1 Hz, t=75 ms). However, the Hess model 
does not reproduce the higher frequency responses 
very well. 

As expected, these results indicate that the 


Repetition 

(K=2,L a =l, 
£=0.7, co = 1 Hz, 
r=75 ms) 

Calculated Kp 
Using System 
Identification 

Calculated Kr 
Using System 
Identification 

1 

0.55 

9.20 

2 

0.51 

6.80 

3 

0.77 

8.06 

4 

0.87 

6.46 

5 

0.89 

5.95 

6 

1.11 

7.59 

7 

0.68 

6.56 

8 

0.89 

8.86 

9 

0.85 

7.61 

10 

0.78 

5.99 

11 

1.05 

7.34 

12 

0.66 

8.43 

13 (outlier) 

0.06 

35.69 

Average±StDev 
(for Repetitions 
1-12) 

0.80±0.18 

7.40±1.09 



Figure 20. Model vs. Subject Tracking 


model changes in pilot control behavior due to the 
values of K p and K r vary with the aircraft dynamics. 
The dependence of the pilot model on aircraft 
dynamics prompts consideration of other techniques 
that could sudden changes in aircraft dynamics in 
real-time. The technique considered has been 
successful in dealing with the changes in aircraft 
dynamics from the flight control perspective. The 
ultimate goal is to develop an algorithm that would 
capture pilot behavior changes due to aircraft 
dynamic changes without a priori knowledge of how 


these dynamic changes occur. The approach outlined 
herein is only an initial step. 

Estimating Pilot Response with a Simple 
Gradient Descent Estimator 

A number of different pilot models with a 
specific structure have been proposed in literature [4- 
7, 16]; however, all of these depend on a priori 
knowledge of aircraft dynamic changes to alter the 
pilot behavior model. The proposed approach is to 
assume the pilot can be represented by a system in 
the linear parameterization form: 

y(t) = ® T <b{x(t)) + S (t) (4) 

where y(t) e □ m , x(f) e □ n are the system output and 

input signals, the ( N x m) matrix 0 e 0 Nxm contains 
the unknown parameters to be estimated, 
0(x(f)) gU n (called the “regressor”) represents the 
N- dimensional vector of chosen basis functions, and 
s(t) eO m denotes the non-parametric uncertainties 
in the system (such as noise, modeling errors, etc.). 
Note that in the equation above both y(t ) and 

0(x(f)) are known quantities. This implies that (4) 
is simply a system of linear equations in terms of the 
unknown 0 which at time t is estimated by 0(f) . 
Then based on the latter, one can predict the 
value of the system output: 

j>(f) = 0 r O(x(f)) (5) 

where j)(f) is called the predicted output at time t. 

Consider a simple gradient estimator to update 
the predictor output y(t) . The basic idea in the 
gradient-based estimation is to update the estimated 
parameters 0(f) in such a way that the prediction 
error e = y(t) - y(t) is reduced. The online 

implementation of the Gradient Estimator is of the 
form: 

4>(0 = -ro(x( 0 )(j>( 0 -^( 0 ) (6) 

where T = > 0 is a symmetric positive definite 

matrix called the estimation gain [17]. 

For pilot model identification, x(f) is the pitch 
attitude commanded by the flight director while y(t) 
is the aircraft pitch attitude response to the pilot stick 


command as the pilot attempts to follow the flight 
director. The update law for the estimates of the 
unknown parameters (6) is a function of the tracking 
error and a regressor chosen a priori. The only 
assumption about the stmcture of the pilot model 
made in this formulation is that it can be represented 
as linear parameterization. 

A sample result for the predicted pilot output 
using the gradient estimator is given in Figure 21 . As 
can be seen in Figure 21, this initial estimate 
(ThetaHatPilot) follows the pilot’s response 
(Theta Pilot) fairly well. Furthermore, this method 
does appear to pick up more of the pilot’s high 
frequency responses. 
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Figure 21. Pilot and Estimator Pitch Response 

The parameters of the gradient estimator appear 
to remain fairly constant from subject to subject with 
the same pitch dynamics (Fig. 22). However, when 
the pitch dynamics change within a subject, the 
current combination of the basis functions 0(x(f)), 

which are held constant across all subjects and all 
dynamics, and the gradient estimator parameter 


Pitch Response for Scenario 5 for Subject 4 



Pitch Response for Scenario 5 for Subject 1 



Figure 22. Pitch Response for Different Subjects 

update law (6) do not do a good job of covering the 
range of dynamics. (Fig. 23). 


It is clear from Figures 21 and 22 that while the 
predictor is doing an adequate job in predicting the 
pilot during some time intervals, i.e., the predictor 
matches the actual pilot response, but it does a poor 
job in others. One issue is the estimation 
methodology as given by the update law (6). The 
simple gradient descent estimator was used as a proof 
of concept. Current research is looking at a variety of 
different approaches to find ones most suitable to 
predicting the pilot in this general model form. 

In this more general model formulation the basis 
functions vector O(x(0) is expected to cover the 
range of dynamics and differences among the pilots; 
the unknown parameter ©(£) changes in a prescribed 
manner to accommodate the differences between the 


Pitch Response for Subject 4 for Scenario 5 



Pitch Response for Subject 4 for Scenario 8 



Pitch Response for Subject 4 for Scenario 22 



Figure 23. Pitch Response for Different Dynamics 

dynamics and pilots based on a specific update law 
(6). This is different from a fixed structure model 
where parameters would change with the dynamics, 
e.g. the Hess model, but would have to be updated 
manually not based on a specific update expression. 

However, it is possible the estimator’s 
parameters ©(£) may need to be tuned to the current 
dynamics which may be estimated using real-time 
system identification techniques [18]. Current 
research is determining if this is necessary and 
feasible. If this is feasible, then pilot model would be 
build on the basis of actual aircraft dynamics. In this 
case, other pilot model structures [4-7, 16] might also 
be suitable for real-time pilot behavior prediction. 


Conclusions 

An experiment was conducted to quantify the 
effects of changing dynamics on a subject’s ability to 
track a signal in order to eventually model an 
adapting pilot to changing aircraft dynamics. The 
research presented here attempts to identify primary 
aircraft dynamics variables that influence changes in 
pilot’s response and produce a simplified pilot model 
that incorporates this relationship. Each run 
incorporated a different set of second-order aircraft 
dynamics representing short period transfer function 
pitch attitude response: damping ratio, frequency, 
gain, zero location, and time delay. 

In the aircraft dynamics, the factors that affected 
the subjects’ ability to track the longitudinal signal 
the most were the time delay, frequency, and zero 
location. Besides the subjects’ RMS error, subjects’ 
CH ratings also showed this effect. Furthermore, this 
increase in RMS error was even more pronounced 
when looking at the control anticipation parameter 
and when considering subjects’ comments regarding 
pilot-aircraft coupling. 

The Hess simplified pilot model using gains 
obtained from system identification techniques shows 
promise in predicting pilots’ control behavior but 
further refinement on the method is needed. Under 
current consideration is a more general form of the 
pilot dynamics model borrowed from flight control 
application dealing with changing aircraft dynamics. 
Several estimation techniques are being explored to 
find most suitable approach to predicting the pilot 
without a priori knowledge of changed aircraft 
dynamics. 

In addition to creating a simplified pilot model 
to serve as a short-term control action behavioral 
predictor, the results of the experiment may be 
utilized in an advisory capacity. Thus, if the aircraft 
dynamics change such that the system becomes more 
sluggish and unpredictable to the pilot, an increase in 
RMS error can be predicted. If a system does in fact 
encounter these factors (established through real-time 
parameter identification), advisory information that 
would aid the pilot in adjusting his piloting approach 
and changing his expectations of aircraft response 
can now be provided reliably by the system. A pilot 
will eventually notice these changes while flying the 
aircraft but a situation awareness/prediction aid based 
on the pilot behavior and aircraft dynamics may help 


tailor pilot’s inputs more quickly so that PIO or an 
upset condition can be avoided. 
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